Towards robust feature selection techniques

Robustness of feature selection techniques is a topic of recent interest, especially in high dimensional domains with small sample sizes, where selected feature subsets are subsequently analysed by domain experts to gain more insight into the problem modelled. In this work, we investigate the robustness of various feature selection techniques, and provide a general scheme to improve robustness using ensemble feature selection. We show that ensemble feature selection techniques show great promise for small sample domains, and provide more robust feature subsets than a single feature selection technique. In addition, we also investigate the effect of ensemble feature selection techniques on classification performance, giving rise to a new model selection strategy.